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A major emphasis in technology education is learning the practices of designers-making informed decisions, 
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trading off achievement of criteria against each other, working in a team, communicating ideas and results, and so 
on. But how can one help students learn such ill-defined and open-ended s ki lls? There have been a variety of 
suggestions made by the best teachers and those who observe and analyze their practices about how to achieve learning 
from project experiences (e.g., Barrows, 1985; Blumenfeld et al., 1991; Bransford et al., 1998; Collins, Brown, & 
Newman, 1989; Monk & Osborne, 2000), and textbooks and innovative modules are being developed based on those 
best practices (e.g., Bell, Davis, & Linn, 1995; CTGV, 1997; Kolodner, Crismond, Gray, Holbrook, & Puntambekar, 
1998; Kolodner, Gray, & Fasse, 2003; Kolodner et al., 2003; Edelson, Gordon, & Pea, 1999, Penner, Lehrer, & 
Schauble, 1998; Reiser et al., 2001). The literatures of cognitive science and the learning sciences inform us about the 
learning of skilled practices (e.g., Bransford, Brown, & Cocking, 1999; Collins et al., 1989; Palincsar & Brown, 1984; 
Scardamalia, Bereiter, & Steinbach, 1984) and make some suggestions about how to promote learning such skills as 
reading and writing. But there has been only a little research devoted specifically to using what we know about skill 
learning to promoting learning of the kinds of skills designers engage in (e.g., Cossentino & Shaffer, 1999; Craig & 
Zimring, 2000; Jansson, 1993; Schon, 1990). 

In this paper, we focus on lessons we've learned from an inquiry into science education about how to promote 
learning of such s ki lls. Many think about science education as education towards learning the 'facts" or content of 
science and how to apply them. But science education that is truly aimed towards scientific literacy focuses as well on 
learning the practices of scientists-designing and carrying out investigations in a replicable way, accurate observation 
and measurement, informed use of evidence to make arguments, explanation using scientific principles, working in a 
team, communicating ideas, and so on. In fact, scientists and designers practice many of the same s ki lls. 

The focus of our endeavor, called Learning by Design™ (Hmelo, Holton, & Kolodner, 2000; Kolodner et al., 
1998, 2003, Kolodner, Gray, & Fasse, 2003), has been to help middle school students (grades 6 to 8; ages 12 to 14) 
learn science content in such a way that they can apply it in new situations and engage skillfully in the practices of 
scientists. Students learn content and skills in the context of engaging in design challenges. For example, they leam 
about motion and forces by spending eight weeks iteratively designing, building, and testing a miniature vehicle and its 
propulsion system that can go a certain distance over a certain terrain. They learn about mechanical advantage by 
designing and building machines for lifting heavy objects. They learn about erosion and water drainage and their 
management in the context of trying to figure out how to keep the dirt on a hill from accreting on a newly built 
basketball court at its bottom. They do this by iteratively designing, constructing, and testing physical models of earth 
formations and of erosion-management approaches. They learn about aquifers, the rock cycle, and rock characteristics 
by producing a geological report and advice on drilling for core samples to an imaginary construction company that is 
bidding for construction of a subway system. Models that they design, construct, and run along the way help them 
understand the rock and water issues the construction company will have to consider. 

In Learning By Design (LBD), the design challenge provides a reason for learning the science content, and 
engaging in the challenge provides a natural and meaningful venue for using both science and design skills. The need 
to make one's design ideas work provides opportunities and reasons for students to identify incomplete and poor 
conceptions of science content and to debug those conceptions; the iterative nature of design provides opportunities to 
apply and test new conceptions; and the collaborative nature of design provides opportunities for team work and the 
need to communicate ideas and results well. 
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Students who have participated in LBD pilot and field tests have learned science content as well or better than 
those learning under more traditional methods. More importantly, though, they have learned many of the skills of 
scientists and designers well enough to collaboratively carry them out quite expertly along with their peers. Our 
analyses show that LBD students engage in collaboration, communication, informed decision making, and design of 
investigations in a far more expert manner than their matched comparisons (Kolodner, Gray & Fasse, 2003). Indeed, 
LBD students in typical-achievement classes perform these skills and practices as well as or better than honors students 
who have not been exposed to LBD, while LBD honors students perform the targeted skills and practices like high 
school or college students. 

We devised Learning by Design based on principles about learning and transfer gleaned from the cognitive 
science and learning sciences literatures. In the process of making our ideas work in middle school classrooms, we've 
learned how to put those principles into practice in learning skilled practices. Five interconnected and overarching 
strategies for learning skills and practices have emerged from our work: 

1. Foregrounding of skills and practices: Cycles of iteratively 
pplying what one is learning, interpreting and explaining one's 
solutions, debugging one's knowledge and skills based on those 
explanations, planning and predicting future use, and then applying 
what one has just figured out; reflection on use is critical. 

2. Practicing: Consistent, continuous, reflective practice of targeted 
skills in contexts consistent with authentic use; undertaken 
individually, in small groups, and publicly. 

3. Establishing need: Giving students a need to use the skills and 
practices as well as reasons to want to work together and leam 
from each other, through challenges that are complex enough to 
require collaboration and that require the targeted s ki lls. 

4. Making recognition of the need to use procedures automatic: 

Through ritualizing the practice of important packages of skills. 

5. Establishing and enforcing expectations: Through establishing 
and sustaining a culture of rigorous thinking and collaboration 
through activities that make the need for such practice clear, and 
then using that culture to promote and support reflective practice. 

These five principles arose from investigations that began by consulting several bodies of literature focused on 
skill learning and transfer, followed by development of a first-pass approach to learning science from design based on 
this literature, and then iterative redesign of the approach based on experiences in the classroom and additional 
literature searches. We have also learned a considerable amount about tactics for implementing these principles. 

This paper continues by defining the terms ’s ki lls" and 'practices" and discussing what it takes to learn them. We 
then present excerpts from two LBD units that illustrate the strategies and tactics we use. Finally, we consider three big 
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issues in making such an approach work: (a) assessing the degree to which students are learning skills and 
practices, (b) preparing teachers for managing a classroom environment such as the one we propose, and (c) taking the 
approach from science to technology classrooms. We concentrate on design decisions we've made in developing our 
LBD units and on their implications for learning skills and practices outside of the LBD context. 

Defining Skills and Practices 

We begin by explaining what we mean by skills and practices. We listed several above: informed decision 
making, identifying and using evidence well to make an argument, designing and running an investigation, working 
with a team, communicating ideas and results. Some skills are specific to a discipline-for example, running a well- 
controlled and replicable physics or chemistry experiment. Others are common across disciplines but may be carried 
out with different standards of practice and using different knowledge, depending on the community carrying them out- 
for example, making informed decisions, working with a team, and communicating ideas and results. 

These skills are part of the defining characteristics of scientific literacy promoted by the science education 
community (American Association for the Advancement of Science [AAAS], 1993; National Research Council [NRC], 
1996) and of the skills and practices identified by the technology education community as important for technological 
literacy (International Technology Education Association [ITEA], 2000). There is a recognized community of skilled 
practitioners (e.g., scientists, engineers, industrial designers) who use these practices from day to day to accomplish 
goals that our society values. 

For example, scientists aim to develop better understandings of the world around us. Their results are accepted or 
challenged by the community based on the skill and logic of those making claims. Are the questions they are asking 
important and sound ones? How well did they design and carry out their investigations? How well did they interpret 
their results? Are they providing sound justifications for those interpretations and their implications? Having one's 
claims accepted requires skills associated with inquiry, investigation, and interpretation, as well as communication 
skills. Inquiry, investigation, and interpretation often require teamwork and, hence, collaboration skills are required as 
well. 


Engineers aim to design artifacts and discover principles governing their design that will allow us to have more 
control over the world. As within the scientific community, the designs and design principles put forth by these 
practitioners are accepted or challenged based on the skill and logic of those making claims. But the questions asked in 
judging the claims are different. Have they focused on an important design need? How well did they define the criteria 
and constraints for achieving that design need? Did they trade off criteria against each other in a way that can be 
justified? In what circumstances does their solution work? Where does it fail to work? What principles can be derived 
from the solution that can be used elsewhere, and how broadly do they apply? Having one's claims accepted requires 
skills associated with clearly specifying the design problem or set of problems, identifying the constraints one is 
working under and prioritizing the criteria to be achieved, understanding the attributes of materials, applying what is 
known, testing one's ideas, explaining failures, iteratively moving toward working products, and extracting principles 
for future work. While judgment of designs is based on how well and broadly they work, designs are conceived in 
conjunction with clients, and communication of ideas is critical to achieving a solution that fulfills the client's needs. 
Construction skills are imperative in making an artifact work. Many of the steps in getting to a working artifact also 
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require teamwork, and, hence, collaboration skills. 

Skills and practices are the complex cognitive, social, and physical actions carried out by members of a 
community. Skills refers to the strategic and tactical procedures an individual carries out-identifying evidence that 
would inform a solution, adapting an old solution to fit a new one, communicating one's reasoning, carrying out a 
procedure in a replicable way. Practices package up the skills that are highly valued by a community and consistently 
carried out by its members, often collaboratively. The better one's expertise in carrying out valued skills, the higher 
one's standing within that community. 

What we aim for in science education is that learners establish at least a working understanding of the skills that 
are valued by science's community of practice, with more advanced students able to carry them out with more fluency 
and over a larger variety of situations. Scientific literacy means having familiarity with and appreciation of the 
practices of scientists, so that, among other things, learners will be able to judge the veracity of claims made by others. 
Technological literacy has similar goals (ITEA, 2000), including familiarity with and appreciation of the practices of 
designers, so that, among other things, learners will be able to use technologies in informed ways. 

There are many science and technology skills that all learners should possess, such as the ability to evaluate 
evidence and use it to make informed decisions. Learners who want to become scientists or engineers need to be able to 
engage more skillfully in the practices of their community. Similarly, any learner who will support or interact with 
those practitioners (e.g., attorneys, craftspeople, information technologists, and so on) will need to gain familiarity and 
a shared working understanding of the skills and practices of both science and technology. 


Table 1. A Selection of Science and Technology Skills and Practices 


Science Skills and Practices 

Technology Skills and Practices 

Understanding a problem and what 
might need to be investigated 

Identifying criteria, constraints, 
problem specifications 

Generating questions that can be 
investigated 

'Messing about" with and 
understanding materials 

Investigation with a 
purpose-experimentation, modeling, 
learning from cases, managing 
variables, accurate observation and 
measuring, seeing patterns, . . . 

Investigation for the purpose of 
application-designing and running 
models, reading and learning from 
case studies, . . . 

Informed decision making, reporting on 
and justifying conclusions 

Informed decision making, reporting 
on and justifying design decisions 

Iteration towards understanding 

Iteration towards a good enough 
solution 

Explaining scientifically 

Explaining failures and refining 
solutions 
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Investigation planning 

Prioritizing criteria, trading them off 
against each other, and optimizing 

Communication of ideas, results, 
interpretations, implications, 
justifications, explanations, principles 

Communication of ideas, design 
decisions, justifications, explanations, 
design rules of thumb 

Teamwork, collaboration across teams, 
giving credit 

Teamwork, collaboration across 
teams, giving credit 


The exemplar skills and practices listed in Table 1 are considered foregrounding skills in LBD and are all listed in 
the science and/or technology standards. Our research group has experience promoting the learning of each. 

Promoting Learning of Complex Skills and Practices: What the Literature Suggests 

The cognitive psychology, educational psychology, developmental psychology, science education, and learning 
sciences literatures provide some basic guidelines about promoting transfer (the ability to use skills or knowledge 
learned in one situation in another the learning didn't specifically target), the development of the ability to recognize 
applicability of a skill and to carry it out skillfully, the role of a community of learners in facilitating learning of 
practices, and the kinds of scaffolding, or support, needed to promote skill learning (see, for example, Bransford et al., 
1999). Together, they advocate several important practices for promoting skills learning. 

First, they advocate deliberate reflective practice of targeted skills (Ericsson, Krampe, & Tesch-Romer, 1993), 
which involves learning in the context of doing that includes monitoring one's own experience of learning, and 
frequent, timely, and interpretable feedback. Deliberate means that the skills are practiced in a context that promotes 
learning; reflective means that their practice is discussed and lessons drawn out from that discussion. Feedback is 
timely when one can use it to determine the quality of one's conceptions. The best feedback comes right after one has 
performed some task, and comes in a form that is easy to interpret (Bransford, et al., 1999; FImelo, et al., 2000; Schank, 
1982, 1999; Schank & Cleary, 1994). When learning in a context of doing, learners take on goals that give rise to a 
need to learn; they fail in gentle and educational ways, motivating question asking and explanation; they have a need to 
try again (iterate), which provides an opportunity for applying and testing their revised conceptions; and they have a 
need to interpret their experiences so as to learn lessons from them (Kolodner, 1993, 1997; Schank, 1982; 1999). 

Second, the literature is specific about reflection that promotes learning. It advocates making the reasoning 
involved in carrying out skills transparent to students. A variety of tactics have been suggested, including modeling by 
the teacher of important skills, articulation by the teacher of his/her reasoning, coaching of learners as they are 
attempting to carry out the skills, and articulation by students of their reasoning (Palincsar & Brown, 1984; 
Scardamalia et al., 1984; Schoenfeld, 1983, 1985, 1991). Learning is enhanced if, in addition, students consider the 
transfer implications of what they are learning. For skills learning, this means anticipating when it would be 
advantageous to carry out a skill in a certain way or how one would carry out the new skill in different circumstances 
(Anderson, Reder, & Simon, 1996; Klahr & Carver, 1988). The case-based reasoning literature suggests three 
additional goals to reflection (Kolodner, 1993, 1997, Schank, 1982, 1999): (a) connecting up one's goals, intentions, 
plans, procedures, results, and explanations of results; (b) explaining results that are different than what was expected, 
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and from those explanations developing new conceptions; and (c) reflection across experiences to extract 
commonalities (and differences). 

The literature warns that learning of practices and skills comes slowly, that deliberate and reflective practice must 
be carried out consistently over a long period of time (e.g., Bransford et al., 1999; Campione, Shapiro, & Brown, 
1995), and that reflection should be aimed toward helping learners develop deep understanding. Concepts and 
strategies being learned need to be experienced across multiple contexts so that learners can extract relevant 
characteristics across contexts and learn a range of applicability conditions. And it is important that students not only 
understand the specifics of particular problems but also learn the abstractions and principles behind those problems, so 
that they will be able to transfer what they are learning to a wide range of analogous situations (Gick & Holyoak, 1983; 
CTGV, 1997; Kolodner, 1997; Singley & Anderson, 1989). Researchers admonish that learners need help identifying 
productive abstractions (Holyoak & Thagard, 1995) and noticing similarities and differences across diverse events, and 
they reassure that experience with a variety of concrete problems coupled with extracting the principles that tie them 
together will result in learners who can think more flexibly (Bransford et al., 1998, Spiro, Feltovich, Jackson, & 
Coulson, 1991). Such practices engage students actively in focusing attention on critical issues, critical features of 
problems, and critical abstractions and principles, and on evaluating their own understanding. 

The literature tells us that learning for transfer is a dynamic and strategic process, that transfer itself includes a 
variety of complex decision making skills-choosing strategies, considering resources, judging applicability, and seeking 
and receiving feedback-and that it depends on active memory processing that re-represents and re-organizes what's 
been learned (Brown, Bransford, Ferrara, & Campione, 1983; Singley & Anderson, 1989). Before someone is ready to 
fully engage in transfer independently, he or she is often able to do the applicability and application part based on 
somebody else's prompting (Gick & Holyoak, 1980; Perfetto, Bransford, & Franks, 1983), and practice engaging in 
transfer seems to be essential to deep and lasting learning. 

The literature suggests that peers can help each other in developing these competencies. Cognitive apprenticeship 
(Collins et al., 1989) advocates orchestrating the classroom so that as students begin to gain expertise in targeted skills 
and practices, they can model for and coach their peers. The approach called Fostering Communities of Learners 
(Campione et al., 1995) advocates not only that students act in ways that promote their peers' learning but also that the 
classroom's ethos should value collaboration and learning together, creating an environment in which students realize 
that they have responsibility for helping each other learn. Problem-based learning (Barrows, 1985; Koschmann, Myers, 
Feltovich, & Barrows, 1994) provides advice about tools to use in the classroom to help students maintain continuity as 
they are solving a problem (a 'whiteboard" is used to record student knowledge, ideas, questions, and plans for moving 
forward), the ways in which students should interact while working on challenges (they should divide up investigative 
activities in such a way that they need each other's results), and roles a teacher should play (e.g., modeling skills, 
explaining how she carried them out, coaching, helping students stay on track, helping students extract out what they 
can learn from their experiences, pushing students to justify their ideas and explain principles they bring for others to 
use). 


The architecture education community adds to this pragmatic advice about setting up the environment to enhance 
the learning of practices (e.g., Schon, 1985), including the role of the studio and the ki nds of presentation opportunities 
that are useful for learning skills. They advocate learning in an environment where one can be immersed in the 
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practices of the community. In a design studio, this means having available the tools and resources that architects 
use, and, as architects do, having available considerable wall space for hanging designs in progress and resources that 
might lead to new ideas. They also promote several kinds of presentations, including critiques, where a designer 
explains his/her design privately to a peer or the teacher for critique, and pin-up sessions, where the design and its 
justifications are presented (usually visually) to a board of experts who comment on it. 

Promoting Skills Learning with Learning by Design™ 

Learning by Design builds on these bodies of work, taking suggestions about classroom practice from the 
cognitive apprenticeship, problem-based learning, and fostering communities of learners approaches, and aiming to 
integrate the kinds of experiences and reasoning about them suggested in the literature. Because the practice of design 
affords practicing nearly all of the skills targeted in the science standards (see Table 1), and because design, 
construction, and trial of working devices affords visible feedback about the designer's intentions and conceptions, 
LBD situates science learning in the context of designing devices. Curriculum units organized around a design 
challenge provide opportunities for scientific inquiry, for applying what is being learned with immediate feedback, and 
for engaging in and learning complex cognitive, social, practical, and communication skills as students leam and apply 
science content. Construction and trial of real devices gives students the opportunity to experience uses of science and 
to test their conceptions and discover the bugs and holes in their knowledge. Figure 1 shows the cycles of activities 
involved in learning from design activities. 
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(Use Science) 


Under sland 
Challenge 




Need to Do 


Present & Share 
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Figure 1: Learning by Design's Cycles. From 'Promoting Transfer through Case-Based Reasoning: Rituals and 
Practices in Learning by Design Classrooms," by J.L. Kolodner, J. Gray, and B.B. Fasse, 2003, Cognitive Science 
Quarterly , 3(2), 119-170. Reprinted with permission. 


A sequence of activities tends to fall out of the cycle. Understanding the challenge comes first (top of the 
Design/redesign cycle) and involves 'messing about" in small groups with materials or devices, and then, as a class, 
engaging in a problem based learning-type 'whiteboarding" session-generating ideas and questions that need to be 
answered, and identifying what needs to be investigated to successfully achieve the challenge. Investigation follows, 
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with students focusing on activities in the Investigate & Explore cycle. In our science classes, this means 
clarifying the question that needs to be answered, making a hypothesis about its answer based on what one already 
knows, designing an experiment, running it, analyzing results, and presenting to the class in a poster session. Poster 
sessions, like whiteboarding, are a whole-class activity. Each group presents its investigation strategies and results, as 
in a poster session at a scientific or technical conference, and peers question group members about their procedures and 
interpretations and provide suggestions and critique. When all groups are finished presenting, the teacher helps students 
look across the different groups' procedures and results and extract out 'design rules of thumb" that the results suggest 
and 'fair test rules of thumb" about investigative procedures that result in trustworthy results. Students return to 
understanding the challenge, specifying the criteria and constraints they now understand to be important to its 
achievement. 

Design/redesign continues with students using the results of investigations to plan their best solution to the 
challenge. Students work in small groups and are asked to be careful to note each of their design decisions and the 
evidence and reasoning justifying each. They present their ideas and their justifications to each other in a 'pin-up 
session," which is similar to a poster session but with a focus on ideas and their justifications. Design rules of thumb 
are revisited, and discussion after the poster session focuses on those and on deriving and reviewing 'evidence rules of 
thumb" that guide the identification and effective use of evidence to justify a decision. 

Students return to their designing, revising their design plans based on suggestions and critiques made by their 
peers, and then they begin constructing and testing their designs, keeping records as they test. This is followed by 
analysis of results and another presentation to the class, this time of what happened when they tried out their design 
ideas. In these presentations, called 'gallery walks," students review their design ideas with their peers, show the device 
they built and what happened when they ran it, and try to explain its behavior. They try to tell the class how they will 
proceed-what they need to learn more about or how they revised their understanding of something and what they will 
do next to their design as a result. But many students are not able to explain their device's behavior by themselves. For 
these students, the gallery walk provides a time for getting help with explaining their device's behavior and figuring out 
what to do next. The gallery walk is a time for engaging publicly in debugging, explaining, and redesigning. Whole- 
class discussion afterwards focuses on revising design rules of thumb; debugging, explaining, and redesigning, and 
creating rules of thumb for doing those well; and then returning to the whiteboard and the challenge and revising 
criteria and constraints as a class. 

Groups continue doing whatever is needed for their designs. Some may need to learn something new and engage 
in additional investigation before continuing through the Design/redesign cycle; some will move on to re -planning, and 
others will move on to constructing their devices better. The teacher calls gallery walks followed by whiteboarding 
several times more as the groups move toward achieving the challenge, but when all groups seem to be on the road to 
success, groups navigate the cycles as needed until designs are completed. 

A set of examples will illustrate. Notice how the principles listed in the introduction are enacted in classroom 
activities. The first example is of an early activity, done approximately one month into the school year, that has as its 
major goal to introduce students to the skills and practices of designing and doing science. The second example is of a 
follow-on activity done one to two months later in the school year that takes advantage of the skills students have 
learned earlier to promote learning of both complicated content and more rigorous use of skills. 
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Example 1: The Parachute Challenge 

The Parachute Challenge is part of LBD's 'launcher unit" for physical science (Holbrook & Kolodner, 2000), 
where students are introduced to the culture of collaboration and scientific reasoning through a variety of short design 
challenges. They've had experiences introducing them to designing, collaboration, and experimentation, and they've 
seen the movie Apollo 13, where scientists and engineers work together at these practices to save the ill-fated Apollo 13 
mission. Students have already been introduced to the variety of presentation and whole-class discussion sessions, but 
the Parachute Challenge provides students their first opportunity moving through the complete LBD cycle as it 
introduces a bit of the science of forces. 

The teacher begins by introducing the challenge. Students have just watched the scene in the movie Apollo 13 in 
which the re-entry module is slowed down by parachutes as it re-enters Earth's atmosphere. They wonder how 
parachutes are able to do that. That scientific question guides the Parachute Challenge. Students learn answers to the 
question by designing their own parachute (using round coffee filters) that can carry two or three washers as slowly as 
possible to the ground. 

Understanding the challenge 

The teacher helps pique the students' curiosity by presenting several demonstrations of coffee filters falling to the 
ground and asking students to predict which configurations of filters will fall faster or slower. (For example, will three 
coffee filters, one inside the other, fall faster or slower than three filters opened up and taped to each other to create a 
large canopy?) Some can predict; some can't; all become curious about why some configurations fall slower than 
others, and all begin to develop hypotheses (e.g., the bigger the canopy, the slower it will fall). The teacher hands 
coffee filters and tape to student groups and asks them to spend 20 minutes 'messing about" with the coffee filters to 
further develop their hypotheses and to develop questions about what affects their rate of fall. Students work in groups 
with three to four members. They group the filters into different configurations and watch to see the relative rate of fall 
of each configuration of filters. The aim here is that they begin to gain some intuitions about the materials they are 
using and the physics that guides the ways those materials behave. Some groups try out many different configurations 
of filters -different shapes, different numbers of filters, and so on. But some groups don't know what to do at this point 
and spend their 20 minutes simply repeating what the teacher has just demonstrated. 

After 'messing about," the class comes together for 'whiteboarding," presenting to each other what they've 
observed, as well as their hypotheses about why it is happening the way it is, the questions they need to answer to know 
for sure if their hypotheses are good ones, and the questions they need to answer to design their best parachute (e.g., 
what size canopy should we use; should we put holes into it?). During this public session, the teacher helps the students 
see what their peers have done while messing about, helps them to articulate questions and hypotheses, and helps them 
turn their initial questions into questions that can be answered through well-controlled experiments (e.g., what effect 
does size of canopy have on rate of fall; what effect do holes have on rate of fall; where should holes be placed for 
good stability?). They might also have a discussion about the ins and outs of good experimentation, reminding 
themselves of what they've learned about experimentation during earlier activities (e.g., the need to vary only one 
variable at a time; the need to run procedures exactly the same way each time). Indeed, there may be a poster on the 
wall with 'fair test rules of thumb" generated during previous activities and with such entries as 'To insure a fair test, 
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make sure to run procedures exactly the same way each time," and 'To insure a fair comparison, keep everything 
the same except for one variable." Both the teacher's demonstrations and messing about activities promote curiosity. 
Following curiosity-promoting activities with whiteboarding provides a public venue for working through the ins and 
outs of asking good questions and permits students who were floundering earlier to see what other groups have done 
and to gain from their experiences. 

Investigate & explore 

After the class agrees on which of the questions generated are the most important ones for success in the 
challenge, each group of students takes responsibility for answering a question and then designs and runs an 
experiment (investigates) to find an answer. It is usually the end of a class period at this point, and for homework, the 
teacher assigns individuals the responsibility of designing an experiment that will assess the effects of their variable. As 
they are designing their experiments, students use a 'Design Diary" page (Figure 2) that prompts them on what to pay 
attention to in designing and running an experiment and collecting data (Puntambekar & Kolodner, 1998). Although 
class discussions provide pointers about good experimentation (the investigative mode students are using), it is the rare 
learner who can use pointers well, even after several discussions and attempts at use. Design diary pages provide 
reminders about the most important points touched on in class discussions. 
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Figure 2: Design Diary Page: My Experiment. From 'Promoting Transfer through Case-Based Reasoning: Rituals and 
Practices in Learning by Design Classrooms," by J.L. Kolodner, J. Gray, and B.B. Fasse, 2003, Cognitive Science 
Quarterly , 3(2), 119-170. Reprinted with permission. 

Students get together in small groups the next day, comparing and contrasting the experiments they have each 
designed and designing an experiment that takes the best of each. One student may have remembered that multiple 
trials are needed while another grappled with which variables to control. It is rare for a single student to consider 
everything needed for a fair test. The teacher makes her way around the room, providing help as groups need it. 

Students spend a day or two designing and running their experiments and collecting and analyzing their data, and 
at the beginning of the third day each group prepares a poster to present to the class. Group members show their 
experimental design, data, and data interpretations, and try to extract advice for the class from their results. Because 
students need each others' investigative results to be successful parachute designers, they listen intently and query each 
other about experimental design and procedures and gathering and interpretation of data. This provides an opportunity 
to discuss the ins and outs of designing and running experiments well. For example, in running parachute experiments, 
it is common for the group investigating the effect of the canopy size on rate of fall to compare parachutes that not only 
are different in canopy size but also in canopy mass (they use three coffee filters for one chute, five for another, and 
seven for another). They might add a rule to the 'fair test rules of thumb" list that points out the need to make sure that 
when sizes are changed, mass remains the same, e.g., 'When varying size of an object in a fair test, be careful not to 
change mass at the same time." Usually, even though students have already had some previous experience designing 
and running experiments, most groups' results are not trustworthy yet, and they redo their experiments and the cycle of 
activities just described. 

When the class agrees that results most groups have come up with are believable (usually after redoing 
experiments), the teacher helps students examine the full set of experiments and experimental results to notice 
abstractions and extract out 'design rules of thumb" (e.g., 'When the number of filters is kept the same but the canopy is 
made bigger, the chute falls more slowly" and 'When the mass of the load is larger, the parachute falls more quickly"). 
Experiments provide opportunities to experience and record phenomena, but students don't necessarily understand why 
those phenomena are happening. To learn the explanations behind these phenomena, the teacher makes some relevant 
reading available about the science content involved in designing parachutes to help explain the rules of thumb derived 
so far-about air resistance, gravity, Newton's Second Law, or combining forces, or about parachutes-and then helps the 
class begin to explain the rules of thumb they've noticed. 

Design planning (using results) 

Next, students plan their parachute designs, using the combination of experimental results the class has produced, 
design rules of thumb extracted, and scientific principles read about and discussed as a class. Generally, the teacher has 
each individual plan his or her design for homework. The group then discusses each individual design decision that has 
been made and agrees on a group design. The teacher makes her way around the room checking that decisions students 
are making are well-justified and not simply based on the loudness or popularity of a group member. 

Each group prepares another poster, this time presenting the group's design ideas along with the evidence that 
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justifies each decision and predictions about how it will perform. They present to their peers in a 'pin-up session." 
During this activity, justifying decisions using evidence and making predictions are the primary foci, and after groups 
present to their peers and consider their peers' questions and suggestions, the class as a whole discusses not only the 
ideas that everyone has presented, but also what it means to identify and use good evidence, and to make predictions. 

Construct & test; Analyze & explain; Present & share 

Students now move to the construction and testing phase, modifying their designs based on discussions in class, 
and then constructing and testing their first parachute. They use another design diary page, this time with prompts 
helping them to keep track of their predictions, the data they are collecting, whether their predictions are met, and 
explanations of why not (Figure 3). 

None of their parachutes work exactly as predicted, sometimes because of construction problems and sometimes 
because of incomplete understanding of scientific principles and the results of experiments. After working in small 
groups to try to explain their results, the class engages in a 'gallery walk," with presentations focused on what happened 
when each group's design was constructed and tested, why it worked the way it did, and what to do next so that it will 
perform better. Explanations are usually quite poor at this time of year, but the teacher helps students state their 
explanations scientifically and calls on others in the class to help. Hearing the explanations students are generating 
allows the teacher to identify misconceptions and gaps in student knowledge, and discussions about those conceptions 
and gaps might follow the gallery walk, along with readings or a short lecture or set of demos to help promote better 
understanding. This gallery walk is also followed by classroom discussion about the set of experiences presented, and 
revisiting and revising design rules of thumb, correcting them and connecting them better to explanations of why they 
work. Discussion also focuses on the explanations students gave, and the class might begin to generate a working 
definition of 'scientific explanation" (e.g., 'Explaining scientifically means providing reasons why something happened 
that use science terminology and include the science principles learned earlier"). 
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Testing My Design 

Name Date 



Each lime you have a design idea, you need lo test it in a fair way and accurate way. Sketch and 
describe your idea, and describe what and how you are testing it. Tell what you observ e and learn. 
Display the data you collect in ways so that others can understand and learn from your work. 


Test for Design # 



Figure 3. Design Diary Page: 


Testing My Design. From 'Promoting Transfer through Case-Based Reasoning: Rituals 
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and Practices in Learning by Design Classrooms," by J.L. Kolodner, J. Gray, and B.B. Fasse, 2003, Cognitive Science 
Quarterly , 3(2), 119-170. Reprinted with permission. 

Iterative redesign 

Students make their way again around the design/redesign cycle, revising their designs, based on explanations 
their peers have helped them develop and the new things they've learned. They construct and test their new designs, 
each time using another 'Testing my Design" page, and they iterate towards better and better solutions until they or the 
teacher decide they have gotten as far as they need to. Discussions after gallery walks often focus on 'fair testing," this 
time not in the context of an experiment but rather in the context of comparing results across different parachute 
designs. Students will not be able to explain why a new design works better than an old one if they have changed more 
than one thing since the earlier test. Nor will they be able to believe their own results if they don't follow the same 
testing procedures each time. Sometimes teams discover these issues as they are testing a later design and report them 
to the class; sometimes a peer notices that a test was done differently than last time or, if results are confusing, asks 
how a test was done and helps a group discover inconsistencies in their procedures. The 'fair test rules of thumb" may 
be consulted and refined, as might the working definition of 'scientific explanation." 

Finishing up 

The activity takes six to ten class periods. At the end, students hold a final gallery walk and a competition and 
compare and contrast across designs to better understand the scientific principles they are learning, going back to the 
rules of thumb to revise and explain them better. They discuss their collaboration experience, their design process, their 
use of evidence, and so on, and revise the rules of thumb and working definitions. Each student writes up and hands in 
a project report, including a summary of the reasoning behind their group's final design and what they've learned about 
collaboration, design, use of evidence, and so on. 

Example 2: The Balloon-Car Challenge 

The Balloon-Car Challenge comes a month to six weeks after the Parachute Challenge and is a sub-challenge of 
our Vehicles in Motion unit. In this unit, students learn about forces and motion in the context of designing a vehicle 
and its propulsion system that can navigate several hills and beyond. The Vehicles unit builds on the knowledge and 
skills learned earlier and helps students develop a deeper understanding of the effects of forces on motion. 

In the early part of the Vehicle Challenge, students have 'messed about" with toy cars and noticed that some 
navigate hills and bumpy terrain better than others; that some start easier than others; and some go farther, faster, 
and/or straighter than others. They've generated a variety of questions about effects of forces on motion (e.g., 'What 
affects how far a vehicle will go?" 'How can we apply enough force to a car to make it go over a hill?" 'What ki nd of 
engine will get a vehicle started easily and keep it going?"). They've also engaged in the first part of the Vehicles 
challenge, the Coaster Car Challenge, where they design and construct a coaster car that can go as far and straight as 
possible after being let go at the top of a ramp. During the coaster car challenge, they focus on two particular forces 
(gravity and friction), on combining forces, and on the skills of explaining behavior of a device scientifically and 
making decisions informed by evidence. As they iteratively redesign their coaster cars, they try to explain why their 
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vehicles are not going as far or straight as they'd like (the force called friction that pushes in a direction opposite to 
the direction the vehicle is going combines with the forces pushing the car forward to slow it down) and how to make 
their cars go farther or straighter (if one can get rid of a force in the wrong direction, there will be more force in the 
right direction; more force in some direction will make the car go farther in that direction), and to understand the 
sources of friction in their vehicles (wheels rubbing on the chassis, axles not parallel to each other). 

The Balloon Car Challenge comes next, and they are challenged to design and build a propulsion system from 
balloons and straws that can propel their coaster car as far as possible on flat ground. They begin by building several 
balloon-and- straw engines and messing about with them. 

Understanding the Challenge 

As during the Parachute Challenge, understanding the challenge involves first messing about, then whiteboarding 
as a class to identify ideas. By now, students have experienced messing about several times, and they are becoming 
adept at it. There's little floundering. They know that they should quickly try out several different ways of using the 
materials and assess how each seems to work. Thus, it is not uncommon during this session to see students trying out 
engines with shorter and longer straws attached, trying out more than one engine, and trying wider and narrower 
straws. After messing about, they gather together for whiteboarding. They eagerly volunteer what they've observed 
(e.g., 'It seems like a wider straw makes the car go farther," 'We attached two straws to our balloon, and it went really 
far."), argue about what they saw and how to interpret it, (e.g., 'Our car seemed to go farther with a shorter straw," 
'Ours went farther with a longer straw," 'I don't think we can compare across those cars because they didn't go exactly 
the same distance off the ramp. We'll need to run fair tests to really know."), try to explain what they observed (e.g., 'I 
think the wider straw makes it go farther because more air comes out of it, and that must mean more force."), and 
identify variables whose effects they want to know about conclusively (e.g., length of straw, number of straws in a 
balloon, diameter of straw, extra engines, bigger balloons, amount of air in the balloon). The teacher helps them 
organize their thoughts on the whiteboards, and then groups each choose a variable whose effects they will investigate 
in an experiment, and for homework, each individual designs an experiment. 

Investigate & explore 

This cycle is similar to what occurs in the Parachute Challenge, but with two differences. First, students have 
already read about how forces combine and have used what they learned to explain how their coaster cars behaved. 
Therefore, they know enough to be able to make predictions based on their scientific knowledge before designing and 
running experiments. Second, because they have more experience with designing and running experiments, more 
groups tend to design and run better experiments now than during the Parachute Challenge. As during that challenge, 
students take a day or two to design and run experiments, collect and analyze data, and prepare to make presentations to 
each other. They present to each other in a Poster Session, followed by a whole-class discussion in which they extract 
design rules of thumb (for balloon engines) from experimental results, attempt to explain those rules of thumb based on 
what they already know about forces, and identify additional things about forces they need to leam. Students also spend 
time learning about the concepts in Newton's Third Law-equal and opposite forces-and use that science to explain how 
a balloon-powered vehicle behaves before going back to the challenge. 
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Design planning 

Upon returning to the Design/redesign cycle, the class briefly revisits the whiteboard and specifies the constraints 
and criteria of this challenge. For homework, each individual designs his or her best balloon-powered vehicle. Then 
each group determines its best design, using what was learned from the experiments. Students tend to design cars with 
two or three or even four balloon engines, and they tend to use wide straws or several narrow ones attached to each 
balloon. They justify these decisions based on the experimental results they've seen and what they know about forces. 
But they don't know yet how the logistics of getting the cars started will need to constrain their designs (it is generally 
too hard for them to manage blowing up more than two balloons without the air escaping before the car is set in 
motion). Groups present their designs to each other in a Pin-up Session, justifying their design decisions. 

The decisions made during Balloon Car design planning tend to be better justified than those made during the 
Parachute Challenge. Justifications tend to refer to both the experiments that have been done and the principles about 
combining forces that were discussed earlier (e.g., 'We decided to use two balloon engines because Group 3's 
experimental results showed that the more balloon engines, the more force will be exerted, and the farther the car will 
go. We didn't use more than two because we couldn't figure out how to blow up more than two balloons at a time. We 
also decided to double the balloons on each engine because Group 4's experiment showed that double balloons make 
the car go farther. We think this is because a double balloon exerts more force on the air inside the balloon, providing 
more force in the direction we want the car to go"). The teacher might point out to the class that their justifications are 
better this time and ask them if they know why, using the results of that discussion to update 'justification rules of 
thumb" (e.g., 'Justifications can refer to experimental results or to scientific principles; if they refer to both, they will 
convince people more than if they refer to just one"). 

Construct & test; Analyze & explain; Gallery walks, Iterative Redesign, Finishing up 

Activities proceed as in the Parachute Challenge. Each group makes minor changes in its design based on 
discussions in the Pin-up Session, and then groups construct their engines, attach them to their vehicles, and test the 
results. Students present in gallery walks and help each other explain their vehicles' behaviors, perhaps reading and 
discussing more about combining forces. They iteratively redesign their engines, in the end holding a competition and 
writing up their designs. 

There are several differences between the enactment of this challenge and the Parachute Challenge. First, students 
show more fluidity in sequencing their own activities. They understand when it is time to share results or ask questions 
of their peers. Whether or not the teacher calls for a gallery walk, students move around the room showing their peers 
what they've learned and asking their advice. Second, students show more fluidity in collaboration. They work better 
together-sharing the work of building and testing their cars fairly automatically, listening better as suggestions are 
made, and making arguments in their groups and justifying them well. Some students in the class emerge as experts in 
different areas, such as knowing how to make a balloon engine that doesn't leak, being able to explain the balloon 
engine's behavior, and so on. Students seek out the experts when they need them. Students also know more content 
now, and they try to use it to predict and explain their vehicle's behaviors. 

Promoting Skills Learning: Guidelines 
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Our pilot and field test results show that LBD students become quite competent at collaboration, communication, 
design, and science skills. Comparisons between matched groups of LBD and non-LBD students show that LBD 
students greatly outperform comparison students in their abilities to design experiments, plan for data gathering, and 
collaboration (Kolodner, Gray & Fasse, 2003). Indeed, mixed-achievement LBD classes outperform comparison honors 
students on these measures. Observations of LBD classes show that LBD students become competent at several design 
skills, including identifying criteria and constraints, making informed decisions, and justifying decisions. The 
differences in the ways students engage in collaboration, science, and design skills are highly visible in the classroom, 
as well as being visible in the formal assessments we administer after the Parachute Challenge and then at the 
conclusion of the Vehicles unit. Students engage enthusiastically and with skill in several important practices. They get 
better at 'talking science," using the vocabulary of forces well, and also at explaining, justifying, and persuading using 
evidence. The get better at 'doing science"-controlling variables, designing procedures, running procedures precisely, 
measuring, observing, recording, and so on. They require each other to be more rigorous, collaborate skillfully and 
willingly, and learn and use the content. 

Where does this increased skill come from? As stated earlier, we believe that five strategies in LBD are 
responsible for these successes: 

1 . Foregrounding of skills and practices 

2. Deliberative and repeated practice 

3. Establishing need 

4. Making recognition of applicability automatic 

5. Establishing and enforcing expectations 

In the following subsections, we try to make clear how these principles from LBD can be applied in technology 
education classrooms. 

Foregrounding the learning of skills and practices through iterative cycles 

The literature on learning tells us that 'deliberate practice" is essential for learning skills, that transferable learning 
needs to focus on understanding, and that learning happens over long periods of time that include numerous practice 
opportunities. LBD foregrounds these essentials of deliberate practice in several ways. First, reflective and interpretive 
components are built into its cycles. The Design/redesign cycle asks students to reflect publicly on their decision 
making after they've made their first set of decisions about their designs. It asks them to interpret results and present 
those publicly after trying out ideas. The Investigate cycle asks students to interpret their results well enough for others 
to leam from them. The public whiteboarding that happens as a part of understanding the challenge provides a venue 
for reflecting on what needs to be learned and whether students know enough to be able to proceed with the challenge. 

Second, the 'how-to's" of skills they are learning are not only discussed, they are also captured in 'rules of thumb" 
lists and 'working definitions" that are displayed on the walls and reviewed, revisited, and revised over time. The 
prominent display of the how-to's of the skills provides a sign of their importance. 

Third, there is a critical focus on these skills during all whole-class discussions and presentations. The teacher 
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models each, the design diary pages give students clues about what to reflect on, students experience and critique 
each other's use of targeted skills as they are making presentations; and discussion following presentations focuses on 
not only the content of presentations but also the skills and practices used in coming up with results. 

Fourth, grading focuses on results as well as the ability of students to articulate how they arrived at results and 
why the decisions they made were good ones. Grading schemes take into account the rules of thumb and working 
definitions displayed on the walls; students are expected to make use of those guidelines in all of their work. 

Fifth, by carrying out the same practices and using the same skills across challenges, students apply the skills they 
are learning in a variety of situations- several times in the context of one challenge and several more times in the context 
of another. Our results showing that students' skills grow over the course of two units are based on students having six 
or more opportunities using each of the skills and reflecting on their use, and we are collecting additional data to see 
whether and how those skills continue to grow with additional units. 

Finally, in an LBD classroom, having the opportunity to explain and reflect on one's reasoning and decisions and 
then refine them is a way of life. We saw that illustrated several times in the examples. During the Parachute 
Challenge, when students couldn't trust each other's results, they asked their peers to redo their experiments, taking 
advantage of principles of fair testing derived from the first time through. When individuals write up the design 
decisions that went into their parachute or balloon-engine designs, they do it based on their group's attempt at such a 
presentation during the final gallery walk. Their individual write-ups take into account the feedback from those 
presentations. 

Frequent individual and public practice of skills 

In LBD, students practice the same skills several times in achieving one challenge, several more times in 
achieving another, and so on. They practice in small groups and publicly during presentation sessions, and they 
individually practice in regular homework assignments as well as in the lab and project write-ups that they do 
individually. Presentations during poster sessions, pin-up sessions, and gallery walks make the reasoning behind group 
decisions transparent to the students, enabling discussions about the how-to's of carrying out practices and skills, advice 
from others on how to do better, and comparing and contrasting different ways of carrying out the same practices. 
When the teacher sees what students are capable of and where they are having trouble, she can tailor discussions and 
future activities to focus on those skills that are more difficult for students. The need to present publicly provides 
reason for reflecting on and interpreting the group's experiences. Having the opportunity to get help from their peers 
provides reason for trying to explain and for figuring out where help is needed. Having the opportunity to help others 
provides an incentive for students to take pride in their work and apply themselves to reflection and interpretation. 
Students have the opportunity to see how others carry out skills, because peers model for each other. If a group gets 
good results and explains how it got those results, others can benefit from both seeing the results and hearing the 
reasoning and procedures that led to them. 

Establishing need 

Public presentations do their job well only if students are motivated to take them seriously as learning 
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opportunities. As professional colleagues, we present to and listen to each other because we know that we can 
become better at what we do through collaboration with others. We have learned that we can give students that same 
motivation if we ask them to engage in challenges that are interesting and engaging but too hard to complete on their 
own, if we introduce them early on to the fact that they can leam from each other, and if we orchestrate the classroom 
so that they need each others' results to be successful. 

Making recognition of applicability automatic 

The literature tells us that the ability to reuse something learned in one situation in another requires being able to 
recognize that the knowledge might be applicable, judge its applicability, and figure out how to apply it, and that 
managing the whole set is difficult. By packaging several s ki lls and practices into well-rehearsed 'ritualized sequences," 
LBD aims to minimize those difficulties (Kolodner & Gray, 2002). If learners recognize that they are in the context of a 
sequence they are familiar with, they can focus on the content of what they are doing rather than putting cognitive 
effort toward figuring out what to do next. LBD's classroom 'rituals" come at expected times in the cycles and have 
well-defined expectations. Through repetition of the cycles, students leam that to understand a challenge, it is important 
to 'mess about" with materials or devices similar to what one is trying to design so as to identify important factors that 
must be taken into account. They leam that a 'whiteboard" that records observations, ideas, and design issues will help 
to organize thoughts in moving toward a solution. They recognize that understanding culminates in being able to define 
the specifications of what one is aiming to design, but also that one can't really define these criteria and constraints until 
one has an understanding of the effects of different variables. Knowing that they will share their results in a poster 
session with others who depend on their results helps them keep in mind the kind of rigor to aim for in their 
investigations. Knowing that they can influence, get help from, and get praise from their peers in a 'pin-up" session if 
they can articulate their reasoning well helps them keep in mind the rigor to aim for in making informed decisions. By 
packaging key sets of skills into ritualized activities where students are aware of how to participate and the benefits that 
accrue, students leam when the skills are applicable and the principles for carrying them out, allowing them to focus 
attention on carrying them out productively. 

Establishing and enforcing expectations through building and sustaining the culture 

There are two kinds of expectations that are woven into LBD's classroom culture: expectations about the rigor that 
will go into the work being done and expectations about student interactions with others. Our classroom observations 
and formal results show that classes that are most successful in learning skills are those where (a) skills are repeatedly 
practiced and deliberated on, with the teacher playing an enthusiastic role in making that happen; and (b) skills are 
introduced early on in such a way that students recognize the importance of doing them well (Kolodner et al., 2003). 

Our 'launcher units" are designed with these principles in mind. In these introductory units, students engage in 
activities that make the need to engage well in skills clear and salient, and then they engage in those same skills again 
and again. For example, in their first activity, students work in small groups to design a bookstand from index cards, 
rubber bands, and paper clips. After showing off their designs in a gallery walk, the teacher gives them a chance to 
redesign their artifacts. Every group's subsequent bookstand works better than the first in some way. But they have also 
copied ideas from each other. Students notice the copying and accuse each other of doing so. The teacher uses this 
opportunity to help the students consider the benefits of collaboration and the need to give credit to others for ideas that 
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are borrowed. Throughout the launcher unit, as students are designing, the teacher helps them notice when they are 
taking ideas from others and makes sure they give each other credit. Students soon begin to give credit and to enforce 
credit giving. The launcher units have activities in them that similarly introduce notions of fair testing and justifying 
using evidence, and that introduce the full set of LBD's cycling and rituals. By the time students get to serious content 
(in Vehicles in Motion), they can participate fairly easily in LBD's rituals and have some idea about when to engage 
important s ki lls. 

Expectations are established through participation in activities where rigor in practicing skills makes results better. 
The teacher establishes rules of thumb through public recognition and friendly enforcement of their use. When tasks are 
engaged in repeatedly and students themselves begin to enforce the established rules of thumb for design and 
experimentation, the stage is set for sustaining a culture that values rigor. 

Challenges to Success 

We've identified three challenges to making skills learning work in our science classrooms: preparing teachers, 
assessment of skills learning, and time. These, we believe, will also be the challenges in promoting productive skills 
learning in technology education classrooms. 

Teacher preparation 

A major focus of our work in designing LBD and LBD units has been on how to make these kinds of practices 
easy for the teacher to carry out. Most teachers are not used to creating the kind of classroom culture that LBD advises, 
nor are they practiced in facilitating the kinds of class discussions LBD suggests. Teachers need time to learn and 
become comfortable with new practices. And there are a variety of teacher challenges inherent in managing a 
classroom of 30 or more children working on projects and creating a classroom culture that values collaboration. 

We've come to the conclusion that there are several necessities we need to include in our summertime LBD 
professional teacher development (Kolodner et al., 2003): (a) teachers need to experience LBD as their students will; 
(b) we need to help them understand the cycles and rituals that are part of LBD, most importantly iteration, doing and 
reflecting, messing about, whiteboarding, pin-up sessions, and gallery walks; (c) we need to help them learn the content 
they are teaching; (d) we need to give them experience constructing and testing everything the students will construct; 
and (e) we need to give them experience trying out their skills with children. We provide all of that in three-week 
summer workshops. After experiencing LBD and inquiry, participants reflect on it as both students and teachers to 
extract what's important. They spend 90 minutes each morning with middle-school students at Georgia Tech's science 
summer camp, followed each day by reflection on the experience and going over the pedagogy, classroom skills, and 
rituals of LBD. Afternoons are spent learning science and completing their experience with the full set of LBD units 
they will be facilitating in their classrooms. 

We've also had to provide plenty of scaffolding for teachers during the school year. We make ourselves available 
by phone and email, and we make visits to classrooms, but have found that perhaps our most powerful materials for 
teachers are the materials and activities we've created for students. The same sequences and ritualized activities that 
make it easy for students to grasp what they should be doing help teachers in the same way. Students know that they 
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should be focusing on justifying design decisions with evidence in pin-up sessions. Teachers know that during 
pin-up sessions, they should focus their facilitation on those skills. The launcher units help teachers ease into their new 
roles. Similarly, design diary pages that help students remember what to pay attention to while designing an 
experiment, testing a design, or making a design decision help teachers know where to direct their attention as they 
guide students. We've created teacher materials to go with the student books, but they use the student materials more 
often. 

The biggest difficulty for teachers has been control. Our observations suggest that willingness to change the way 
one controls a class is a key to success. Teachers can't always facilitate LBD well right away, but if they've bought in to 
what could be in the classroom, their classes thrive, and students and teachers leam together. 

Assessment of Skills Learning 

The tradition in classrooms is to assess students based on the results they produce. But this goes counter to what's 
needed if we want students to learn skills. We must differentiate good use of skills that led to not-so-great results from 
mediocre use of skills where luck or previous knowledge or ingenuity allowed good results to be accomplished 
anyway. It's difficult to assess the skills learning of individual students when students are mostly doing their work in 
small groups. We've found that what's needed is a way to assign grades that takes both results and skills used to get to 
those results into account and that takes into account both the abilities of individuals to perform as individuals and the 
ability of individuals to contribute to group capabilities and accomplishments. Our best teachers assess a combination 
of individual and group work, and they use formal and informal assessments. Some give tests, and some of them find 
that they collect enough evidence of learning from work the students are doing from day to day that there is no need to 
give tests. Posters and design diary pages that groups work on in the context of achieving design challenges, for 
example, provide solid evidence of a group's capabilities. They contain a record of the group's deliberations -the 
investigations they did and what they learned from them, their ability to analyze data, their ability to make informed 
decisions and justify them well, their ability to explain, and so on. Individual write-ups of experimental work and final 
project reports provide evidence of individual understanding and capabilities. It is also possible to gauge student 
capabilities by giving them additional 'performance tasks" to work on-a smaller task than a full design challenge that 
requires the same skills or knowledge. In our Vehicles unit, for example, we ask students to apply what they've learned 
about forces to the design of a vehicle for Antarctic exploration and to justify each of their recommendations using 
evidence from experiments done and science principles discussed in class. The rules of thumb lists and working 
definitions of skills created by the class can serve as mbrics for assessing the extent to which students are skillfully 
using the skills they are learning. If rules of thumb associated with fair testing advise on how to control variables and 
measure precisely, for example, then student assignments can be graded according to the extent to which they follow 
those rules of thumb. 


Time 


Finally, time on task is a big challenge for the middle school science teachers with whom we work. Most teachers 
are used to covering something and moving on, and most administrators, parents, and students have that expectation as 
well. Local and national standards are being set that ask teachers to cover a huge amount of content without real depth. 
Yet deep learning, whether of content or of skills and practices, requires time, consistent repeated use, and reflection. 
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When time is taken to leam some set of skills or content deeply, there is little time left for covering the breadth of 
content or skills required by a set of standards. We've found no easy way around this for science education; the system 
is not on the side of deep learning right now. But it may be easier in technology education where there is agreement 
that skills learning is essential. A curriculum that focuses on the same set of skills over several different domains of 
application should have the time and variety in it to insure productive skills learning if skills learning is foregrounded 
and ritualized in the ways suggested, if important skills are consistently practiced individually and in public in settings 
authentic to their use, and if expectations of rigor and group work are set and enforced as part of the culture of the 
classroom. 
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Mike Ryan, and Lisa Prince. Our funding for this work came from the National Science Foundation, the Woodruff 
Foundation, and Georgia Tech's College of Computing. 
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